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Abstract Density evolution (DE) is one of 

the most powerful analytical tools for low-density 
parity-check (LDPC) codes on memoryless binary- 
input/symmetric-output channels. The case of non- 
symmetric channels is tackled either by the LDPC 
coset code ensemble (a channel symmetrizing argu- 
ment) or by the generalized DE for linear codes on 
non-symmetric channels. Existing simulations show 
that the bit error rate performances of these two dif- 
ferent approaches are nearly identical. This paper 
explains this phenomenon by proving that as the min- 
imum check node degree d c becomes sufficiently large, 
the performance discrepancy of the linear and the 
coset LDPC codes is theoretically indistinguishable. 
This typicality of linear codes among the LDPC coset 
code ensemble provides insight into the concentration 
theorem of LDPC coset codes. 

I. Introduction 

Low-density parity-check (LDPC) codes have found many 
applications in cellular networks, magnetic/optical storage de- 
vices, and satellite communications, due to their near-capacity 
performance and the embedded efficient distributed decoding 
algorithms, namely, the belief propagation (BP) decoder 0. 
For binary-input/symmetric-output (BI-SO) channels, the be- 
havior of the BP decoder and the decodable noise threshold 
can be explained and predicted by the density evolution (DE) 
method, which traces the evolved distribution on the log- 
likelihood ratio (LLR) message used in BP 0. Additional 
references on LDPC codes can be found in 0. 

Although the classical DE does not apply to binary- 
input/non-symmetric-output (BI-NSO) channels, in practice, 
LDPC codes are applicable to BI-NSO channels as well and 
near capacity performance is reported 0. Rigorous analyses 
of BI-NSO channels are addressed either by the coset code ar- 
gument (namely, a channel-symmetrizing argument) or 
by the generalized DE for linear codes on BI-NSO channels 0. 

A coset code consists of all sequences x of length n satisfy- 
ing 

Hx = s, (1) 

for some fixed, coset-defining syndrome s, where H is a fixed 
parity-check matrix of dimension (n(l — R)) x n and R is the 
rate of this coset code. When s = 0, corresponds to a lin- 
ear code. It has been shown in that for sufficiently large n, 
almost all s £ {0, l} n ( 1-i? ) and almost all H drawn from the 

1 This research was supported in part by the Army Research 
Laboratory under Contract DAAD-19-01-2-0011. 



cquiprobable bipartite graph ensemble, the codeword-averaged 
performance can be predicted by the coset-code-based DE 
within arbitrary precision. If one further assumes that there is 
a common independent, uniformly distributed bit sequence ac- 
cessible to both the transmitter and the receiver, then a coset- 
code-averaged (syndrome-s-averaged) scheme can be obtained 
as in Fig-Pla'), This coset-code-averaged scheme in Fig. 0a) is 
equivalent to a linear LDPC code on the symmetrized channel 
as demonstrated in Fig. 0b) , of which the error-probability 
is codeword-independent. On the other hand, the general- 
ized DE in analyzes the codeword-averaged performance 
when linear codes plus BI-NSO channels are considered as 
in Fig. 0c). It is shown in that the necessary and suffi- 
cient stability conditions in both schemes (Figs. 0b) and0c)) 
are identical. Monte Carlo simulations based on finite-length 
codes (n = 10 4 ) further show that the codeword-averaged 
performance in Fig. 0c) is nearly identical 1 to the perfor- 
mance 2 of Fig. 0b) when the same encoder/decoder pair is 
used. The above two facts suggest close a relationship between 
linear codes and the coset code ensemble. 

This paper addresses this phenomenon by proving that for 
sufficiently large minimum check node degree d c , the asymp- 
totic performance (and behavior) of linear LDPC codes is the- 
oretically indistinguishable from that of the LDPC coset code 
ensemble. In practice, the convergence rate of the thresholds 
of these two schemes is very fast (with respect to d c ) . For mod- 
erate d c > 6, the discrepancy of the asymptotic thresholds 3 
for the linear codes and the coset code ensemble (Figs. 0b) 
and 0c)) is within 0.05%. Our result shows the typicality 
of linear codes among the coset code ensemble. Besides its 
theoretical importance, we are then on solid ground when 
simulating the codeword-averaged linear code performance by 
assuming the all-zero codeword in the coset code ensemble. 

II. Formulation 
A. Code Ensembles 

The coset code ensemble is based on 01, where the coset- 
defining syndrome s is uniformly distributed in {0, l} n ( 1 ~ ii ' 
and H is from the equiprobable bipartite graph ensemble. To 
be more explicit, the equiprobable bipartite graph ensemble 
C n (d v ,d c ) is obtained by putting equal probability on each 
of the possible configurations of the regular bipartite graphs 
with the variable node degree d v and the check degree d c , and 
by the convention that Hj t i, the (j, i)-ih entry of the parity 

x That is, it is within the precision of the Monte Carlo simulation. 

2 In Fig.|TIb), the error probability is codeword-independent so 
that the all-zero codeword can be assumed, which drastically simpli- 
fies the computation of taking the average over the entire codebook. 

3 The asymptotic thresholds (with respect to n) are obtained by 
DE/generalized DE rather than Monte Carlo simulation. 
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Fig. 1: Comparison of the approaches based on the coset code ensemble and on codeword averaging. 



check matrix H, equals one iff there is an odd number of edges 
connecting variable node i and check node j. We can also 
consider irregular code ensembles C n (X,p) such that A and p 
denote the finite order edge degree distribution polynomials 

Mz) = S ^2,\ k z k ~ 1 

k 

p( z ) = '52 phzk ~ 1 > 

k 

where At, or p k is the fraction of edges connecting to a degree 
k variable or check node, respectively. Further details on the 
equiprobable bipartite graph ensemble can be found in [3]. 

If we hard- wire s = and still let H be drawn from C n (X, p), 
we then obtain the traditional linear LDPC code ensemble. 

B. The Classical Density Evolutions 

In this paper, we consider only the BP decoder such that the 
passed message m corresponds to the LLR m — In p[^|^Z°j ■ 
The detailed representation of the variable and check node 
message maps is as follows. 
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For BI-SO channels, the probability density of the messages 
in any symmetric message passing algorithm is codeword in- 
dependent, by which we mean that for different transmitting 



codewords, the densities of the messages are of the same shape 
and differ only in parities. Let P^ denote the density of the 
LLR messages from variable nodes to check nodes during the 
Z-th iteration given that the all-zero codeword is being trans- 
mitted. Similarly, Q"> denotes the density of the LLR message 
from check nodes to variable nodes assuming the all-zero code- 
word. The classical DE 3] derives the iterative functionals on 
the evolved densities as follows. 
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where "®" denotes the convolution operator and all scalar 
multiplications in A(-) and p(-) are replaced by convolutions 
as well. The operator V transforms the distribution on R into 
a distribution on GF(2) xR + based on the measurable function 
7(-). T" 1 represents the corresponding inverse transform. 



C. Generalized DE and the Coset-Code-Based Approach 

For BI-NSO channels, the error-protection capability is code- 
word dependent and we cannot assume that the all-zero 
codeword is transmitted. This difficulty is circumvented by 
the codeword-averaged approach in which we trace pairs of 
evolved densities, ((P« (0), P m (1)) and ((Q (0 (0), Q (0 (l)) , 

where P' l '(x) denotes the distribution of the LLR message 
rn — In from the variable node to the check node 

during the Z-th iteration, given that the transmitting bit at 
the source variable node is x. Q^'(x) denotes the distribu- 
tion of the LLR message m — In p^j^Z^j from the check node 
to the variable node during the Z-th iteration, given that the 
transmitting bit at the destination variable node is x. The 
generalized DE for linear codes on BI-NSO channels can then 



be stated as follows. 

P (i) O) = P (0) (a;) ® A (Q (i_1) (xfj 
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The proportion of incorrect variable-to-check messages in the 
Z-th iteration can be computed by 

Pe'unear ~ J I g J ^ ^ 

By iteratively computing (p®(0), P®(1)\ and checking 

whether p^\ inear converges to zero, we can determine whether 
the channel of interest is asymptotically decodable when a suf- 
ficiently long linear LDPC code is applied. Further discussion 
on the generalized DE can be found in pjj- 

Let (•) denote the average operator such that (/) := 
/(o)+/ W _ Q ne can eas iiy show that the DE corresponding 
to Fig. H^b) becomes 



>(0 



i coset 



)(!-!) 
coset 



P w ) ® A i 

LWp(r(p<£?))), 



(4) 
(5) 



and the proportion of incorrect messages is 
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One can check whether p^ coset converges to zero to determine 
whether the channel of interest is asymptotically decodable 
with a sufficiently long coset code ensemble. Further discus- 
sion on the coset-code-based approach can be found in [Sj. 

III. Typicality of Linear LDPC Codes 

It was conjectured in [7j that the scheme in Fig. ^c) 
should have the same/similar codeword-averaged performance 
as those illustrated by Figs.^a) andQ^b). To be more precise, 
the question is whether for the same channel model (namely, 
for the same initial distribution pair (p (0) (0), P (0) (1) 
are able to show 



lim p 
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t = 0- 



This paper is devoted to the above question. We can answer 
immediately that the performance of the linear code ensem- 
ble is very unlikely to be identical to that of the coset code 
ensemble. However, when the minimum check node degree 
rfc.min := {k £ N : pk > 0} is relatively large, we can prove 
that their performance discrepancy is theoretically indistin- 
guishable. In practice, the discrepancy of decodable thresh- 
olds of the linear and the coset code ensemble is within 0.05% 
for a moderate d c ,min > 6. 

It is clear from that for linear codes, the variable node 
iteration involves convolution of several densities having the 
same x value. The difference between Q ( ' _1) (0) and Q (i_1) (l) 
is thus amplified after each variable node iteration. It is very 
unlikely that the decodable threshold of linear codes (obtained 
from and JjJ) and the decodable threshold of coset codes 
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Fig. 2: Density evolution for z-channels with the linear code 
ensemble and the coset code ensemble. 



(obtained from Q and JjJ) will be analytically identical after 
the amplification during the variable node iterations. Fig. [5] 
demonstrates the traces of the evolved densities for the regular 
(3,4) code on z-channels. 4 With the one-way crossover proba- 
bility being 0.4540, the generalized DE for linear codes is able 
to converge within 179 iterations, while the coset code ensem- 
ble shows no convergence within 500 iterations. This demon- 
strates the possible performance discrepancy, though we do 
not have analytical results proving that the latter will not con- 
verge after more iterations. Table compares the decodable 
thresholds such that the density evolution enters the stability 
region within 100 iterations. We notice that the larger d Ct7n i n 
is, the smaller the discrepancy is. This phenomenon can be 
characterized by the following theorem. 

Theorem 1 Consider BI-NSO channels and a fixed pair of 
finite-degree polynomials A and p. The shifted version of the 
check node polynomial is denoted by pA = x A ■ p where A 6 
N. Let P^oset denote the evolved density from the coset code 
ensemble with degrees (A, pa) (obtained from 01) and 131 ). and 
(P^) = |Ei=o l^^'t 1 ) denote the averaged density from 
the linear code ensemble with degrees (X, Pa) (obtained from 

H|) and {$)). Then, for any l 6 N, limA^oo (P (0 > = P® eet 
in distribution for all I < lo, with the convergence rate being 
O (const A ) for some const < 1. 



Corollary 1 (The Typicality on Z-Channels) Define 

Pl^O,linear ■= SUp { Pl^ > I lim P® lin ear = f 
k I— >oo ' J 

and pt^o.coset := suplpi^o > : lim p^ cosei = o\ . 

K I— *oo J 

For any e > 0, there exists a A £ N such that 

>0, linear ~ Pl->0, coset I < e - 



4 A z-channel is a binary-input/binary-output channel such that 
only bit value 1 may be contaminated with one-way crossover prob- 
ability pi^n. Bit value will always be received perfectly. 



Table 1: Threshold comparison p* y Q of linear and coset LDPC codes on Z-channcls 
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Namely, the asymptotic decodable thresholds of the linear and 
the coset code ensemble are arbitrarily close when the mini- 
mum check node degree d c>m i n is sufficiently large. 

Similar corollaries can be constructed for other channel models 
with different types of noise parameters. For example, the 
a* in the binary-input additive white Gaussian channel, the 
A* in the binary-input Laplace channel, etc. The proof of 
Corollary Q] is in Appendix 1X1 

Proof of Theorem Since the functionals in J!jJ and 
are continuous with respect to convergence in distribution, 
we only need to show that V/ G N, 



^ r- 1 p r 
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where = denotes convergence in distribution. Then by in- 
ductively applying this weak convergence argument, for any 
bounded lo, liniA^oo (P^) = PcJset m distribution for all 
I < lo. Without loss of generality, 5 we may assume pa = x A 
and prove the weak convergence of distributions on the do- 
main 

7 (m) := (l {m < 0} ,lncoth|y|) = ( 7l , 72 ) G GF(2) x R+, 
on which the check node iteration becomes 

lout A = 7»n,l + 7in,2 + ■ ■ ■ + 7in,A- 

Let Pq denote the density of 7in(m) given that the distribution 
of m is P (!_1) (0) and let P{ similarly correspond to P (i_1) (I). 
Similarly let Qq,a an d Q'i.a denote the output distributions on 
Tout, a when the check node degree is A + 1. It is worth noting 
that any pair of Qo,a an d Qi,a can De mapped bijectively 
back to the LLR distributions Q a_1) (0) and 

Let $ F /(fe,r) := E P , {(-l) fe71 e"' 72 } ,Vfc G N,r 6 R, de- 
note the Fourier transform of the density P' . Proving JSJ is 
equivalent to showing that 
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However, to deal with the strictly growing average of the limit 
distribution on the second component of 7ou t,A, we concen- 
trate instead on the distribution of the normalized output 
We then need to prove that 
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5 We also need to assume that Vx, P" 1 )(a;)(m = 0) = so that 
In coth | 21 1 £ M+ almost surely. This assumption can be relaxed 



by separately considering the event that 
{!,■•• ,dc- 1}. 



for some i £ 



We first note that Q ,a is the averaged distribution of 
7out,A when the inputs ji n ,i are governed by P fl ~ 1 \xi) with 
53i=i !C « = 0- Similarly Q'i.a is the averaged distribution of 
7out,A when the inputs ~fi ni i are governed by P' !_1 '(xi) with 
S»=i = !• From the above observation, we can derive the 
following iterative equations: VA G N, 



a;=0,l 



i=0,l 



By induction, the difference thus becomes 



$ p ,(fc,£)-$ p ,(fc,£) 



(7) 



By Taylor's expansion and the channel decomposition argu- 
ment in we can show that for all fc G N, r G R, and for all 
possible Pq and P{, the quantity in Q converges to zero with 
convergence rate O (const A ) for some const < 1. A detailed 
derivation of the convergence rate is given in Appendix [E] 
Since the limit of the right-hand side of J7J is zero, the proof 
of weak convergence is complete. The exponentially fast con- 
vergence rate O (const A ) also justifies the fact that even for 
moderate d c ,min (e.g. d c ,min > 6), the performances of linear 
and coset LDPC codes are very close. ■ 

Remark 1: Consider any non-perfect message distribution, 
namely, 3xo £ {0, f} such that P*-' -1 ' (a;o) / <5oo, where S mo is 
the Dirac delta measure centered on mo. A persistent reader 

may notice that Vx, liniA->oo Q^ l ~^ (x) = So- That is, as A 
becomes large, all information is erased after passing a check 
node of degree A. If this convergence (erasure effect) occurs 
earlier than the convergence of Q^ _1 - ) (0) and Q^ -1 '(l), the 
performances of linear and coset LDPC codes are "close" only 
when the corresponding codes are "useless." 6 To quantify the 
convergence rate, we consider again the distributions on 7 
and their Fourier transforms. For the average of the output 



6 To be more precise, they correspond to extremely high-rate 
codes and the information is erased after every check node iteration. 



distributions (^Q 1 - 1 we have 
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By Taylor's expansion and the channel decomposition argu- 
ment, one can show that the limit of JjJ exists and the con- 
vergence rate is 0(A~ ). (A detailed derivation is included 
in Appendix [5]) This convergence rate is much slower than 
the exponential rate O (const A ) in the proof of Theorem IT1 
Therefore, we do not need to worry about the case in which 
the required A for the convergence of Q' i ~ 1 '(0) and Q" _1 '(l) 

is excessively large so that Va; £ GF(2), Q^ _1 '(a;) ft! <5o. 

Remark 2: The intuition behind Theorem [Tl is that when 
the minimum d c is sufficiently large, the parity check con- 
straint becomes relatively less stringent. Thus we can approx- 
imate the density of the outgoing messages for linear codes 
by assuming all bits {a;i} ie / 1 ... l£ j c _2} involved in that partic- 
ular parity check equation are "independently" distributed in 
{0, 1} rather than satisfying x i = x i which leads to the 
formula for the coset code ensemble. On the other hand, 
extremely large d c is required for a check node iteration to 
completely destroy all information coming from the previous 
iteration. This explains the difference between their conver- 
gence rates: O (const A ) versus ©(A" 1 ). 

Fig. |3 illustrates the weak convergence predicted by The- 
orem and depicts the convergence rates of Q' !_1 '(0) — ► 
Q (i_1) (l) and S£ ' 



'(Q)+Q' 
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IV. Conclusions 

The typicality of the linear LDPC code ensemble has been 
proven by the weak convergence (w.r.t. d c ) of the evolved den- 
sities in our codeword-averaged density evolution. Namely, 
when the check node degree is sufficiently large (e.g. d c > 6), 
the performance of the linear LDPC code ensemble is very 
close to (e.g. within 0.05%) the performance of the LDPC 
coset code ensemble. This result can be viewed as a com- 
plementing theorem of the concentration theorem in [Corol- 
lary 2.2 of UJ], where a constructive method of finding a typ- 
ical coset-defining syndrome s is not specified. 7 

Besides the theoretical importance, we are then on a solid 
basis to interchangeably use the linear LDPC codes and the 
LDPC coset codes when the check node degree is of moderate 
size. For instance, from the implementation point of view, 
the hardware uniformity of linear codes makes them a supe- 
rior choice compared to any other coset code. We can then 
use fast density evolution |9] plus the coset code ensemble to 
optimize the degree distribution for the linear LDPC codes. 
Or instead of simulating the codeword-averaged performance 



7 Our result shows the typicality of the all-zero s when d c is 
sufficiently large. The results in [5], on the other hand, prove that 
a very large proportion of s is typical but may or may not be one 
of them. 



of linear LDPC codes, we can simulate the error probability 
of the all-zero codeword in the coset code ensemble, in which 
the efficient LDPC encoder |10| is not necessary. 

Appendices 

A. Proof of Corollary [l] 

We prove one direction that 

* _* 

Pi — *0 , linear > Pi — '0, coset ^ - 

The other direction that p*^o, coset > Pi— o linear ~ e can be 
easily obtained by symmetry. One prerequisite of the following 
proof is that both the linear code and the coset code have the 
same stability region pjj- 

By definition, for any e > 0, we can find a sufficiently 
large lo < oo such that for the one-way crossover probability 



Pi-o 



: Pi — 0, coset 



■C P 



is in the interior of the stability 



region. We first note that the stability region depends only 
on the Bhattacharyya noise parameter [3], which is a con- 
tinuous function with respect to convergence in distribution. 
Therefore, by Theorem U there exists a A £ N such that 
^p( ; o)^ j s a i so m t ne stability region. By the definition of the 

stability region, we have limj^oo p^\ inear = 0, which implies 
Pi^o, linear ^ Pi— 0- The proof is thus complete. 

B. The Convergence Rates of (JJJ and (JHJ 

For Q, we will consider the cases k = and k = 1 sepa- 
rately. By the binary asymmetric channel (BASC) decom- 
position argument, namely, all binary- input non-symmetric 
channels can be decomposed as the probabilistic combina- 
tion of many BASCs, we can limit our attention to sim- 
ple BASCs rather than general Bl-NSO channels. Suppose 
(pV- 1 ") (0), P^-V corresponds to a BASC with crossover 
probabilities eo and t\. Without loss of generality, we may 
assume eo + ei < 1 because of the previous assumption that 
Vx G GF{2),P (l ~ 1) (x)(m = 0) = 0. We then have 



<S> P ,(k, -) = (1 - ei ) e '*» 1=15=5 + (-l) K eie 
By Taylor's expansion, for k — 0, @ becomes 
'$p,(0,i)-$ P( (0,i) x A 
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= 21 i 
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which converges to zero with convergence rate O (0(A) A ) 
For k = 1, we have 



(9) 



where 
/-(eo,ei) 



(1 - eo + ei)ln (1 + e - ei)ln 

1 — eo — ei 1 — eo — ei 
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Fig. 3: Illustration of the weak convergence of QV-^iO) and Q^^l). One can see that the convergence of 1 '(0) and Q'' ^(1) is 



faster than the convergence of — (°)+Q Hi anc j $ 



@ converges to zero with convergence rate C(const A ), where 
const satisfies |ei — eo| < const < 1. Since the convergence 
rate is determined by the slower of the above two, we have 
proven that Q converges to zero with rate C(const A ) for some 
const < 1. 

Consider (JHJ. By the assumption that the input is not 
perfect, we have max(eo,ei) > 0. For k = 0, by Taylor's 
expansion, we have 



* p ,(0,£) + *p/(0,£ 



1 + 



(10) 



where 
/+(eo,ei) 

:= (1 - e + ei) In h (1 + e - d) In 

V 1 — eo — ei 1 — eo — ei 

The quantity in HIGH converges to 

e »(f )/+( e i. £ 2) 

with rate O (A -1 ) . For k = 1, we have 
$ P >(1, j) + $ P{ (l, j) j A 



= (l-eo-ex) 



. t- , 1 — en+ e l 

ixlni y i i-^-ln-r— 



which converges to zero with rate O ((1 — eo — ei) A ) . Since 
the overall convergence rate is the slower of the above two, we 
have proven that the convergence rate is O (A -1 ) . 
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